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Abstract 

A great deal of work has been generated on feedback in teams and has shown that giving 
performance feedback to teams is not sufficient to improve performance. To achieve the 
potential of feedback, it is stated that teams need to proactively process this feedback and 
thus collectively evaluate their performance and strategies, look for alternatives, and 
make clear decisions about ways to tackle their task. This concept of team reflexivity has 
been commonly described as a sequence of behaviours, which relative importance has not 
been demonstrated. Further, empirical research investigating the dynamic aspects of 
reflexivity has been scarce. This study sought to explore how reflexivity evolves over time 
and at which moments of the team interaction it is related to team performance. Thirty-two 
student dyads participated to a cognitively complex task (flight simulation) over four 
performance episodes comprising action phases followed by transition ( feedback) phases. 
High interdependence between participants (pilots and co-pilots) was ensured through the 
distribution of complementary knowledge in the dyads. The results showed that teams 
seldom engaged in full cycles of reflective behaviours. When looking into individual 
behaviours, teams exhibited more reflective behaviours during action over time, while 
their reflective behaviours during feedback did not change, demonstrating a suboptimal 
feedback processing as time goes by. Additionally, it was demonstrated that teams were 
capable to learn from their past and act upon feedback to better subsequent team 
performance but also that initial performance acts as a trigger to future reflective 
behaviours. 
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1. Introduction 

Small group work has gradually progressed to being one of the dominant approaches in the domain 
of learning and instruction and professional development (e.g., Kirschner, 2009). Collaborative learning is 
one of the most successful and widespread instructional practice implemented in schools and universities 
(e.g., Dillenbourg, 1999, Johnson & Johnson, 1992). Similarly, work teams have become a central element in 
the functioning of organisations in many domains (e.g., health care, military, and aviation) (Salas, Stagl, 
Burke, & Goodwin, 2007). Both professional teams and learning teams face similar challenges inherent to 
collaboration and joint understanding (Barron, 2000; Jarvela, Volet, & Jarvenoja, 2010). Specifically, in both 
environments, interdependent team members need to interact and communicate effectively, share knowledge 
and experiences, and capitalise each other’s skills and resources to successfully complete a common task 
(e.g., Johnson, Johnson, & Stanne, 2000; Salas, Dickinson, Converse, & Tannenbaum, 1992). Crucially, 
recent work has shown that teams that engage in team learning processes and learn how to work effectively 
are more likely to succeed (e.g., Dochy, Gijbels, Raes, & Kyndt, 2014; Van der Haar, Segers, & Jehn, 2013, 
Veestraeten, Kyndt, & Dochy, 2014). Team learning has been defined as “an ongoing process of reflection 
and action” (Edmondson, 1999, p.353) during which teams reflect on their own prior activities and 
consequently plan adjustments for future practice (see Decuyper et al., 2010, for a review). Although 
scholars in these areas have tended to remain isolated within their own disciplines despite obvious overlaps 
in research interests, they generally agree that team learning processes do not occur naturally (Johnson & 
Johnson, 1992; Rummel & Spada, 2005; Sims, Salas, & Burke, 2005). The awareness that not all teams 
learn, and as a consequence may reach substandard group performance, raises the need to outline deliberate 
interventions to build learning in teams. More and more, new research interests focus on what can be done to 
leverage learning in teams and improve their performance (e.g., Decuyper, Dochy, & Van den Bossche, 
2010; Salas, Stagl, & Burke, 2004). Despite these renewed efforts, it seems that potential leverage points 
(such as training or the provision of feedback) calibrated for teams need to be better specified and validated 
(Kozlowski & Ilgen, 2006). 

Giving teams feedback on their team process and performance has been identified as a leverage point 
that shapes team learning and can improve team performance (Gabelica, Van den Bossche, Segers, & 
Gijselaers, 2012; Johnson & Johnson, 2002; London & Sessa, 2006; Phielix, Prins, & Kirschner, 2010). In 
school and beyond, teams need feedback to monitor and regulate their work (Hattie & Timperley, 2007). 
Previous theoretical work on feedback provided by external agents at the team level of analysis (e.g., 
Goodman, Wood, & Hendrickx, 2004; London & Sessa, 2006) suggests that to achieve changes in team 
learning and performance, teams need to process received feedback, be receptive to this feedback, 
understand its value, and actively engage in collaborative activities during which they use feedback cues to 
make improvements. Nevertheless, empirical work on the value of active feedback processing to the mere 
reception of feedback in teams has considerably lagged behind theoretical development (Hattie & Timperley, 
2007). This feedback processing has yet to be empirically examined (Gabelica et al., 2012). More 
specifically, the more interesting question about feedback effectiveness is rather how learning naturally 
happens during the team feedback process and how effective are these learning processes (e.g., Adcroft, 
2011). Moreover, previous work in both team and collaborative learning research leaves much about the 
dynamics of feedback processing in teams unspecified, such as 1) how do teams respond to repeated 
(external) feedback in dialogue over the course of ongoing activities, and 2) when (i.e., at which point in 
time) are these behaviours related to effective learning and performance. There is a general agreement across 
disciplines that we should consider feedback loops in which behavioural changes resulting from each cycle 
are inputs in cycles that follow (e.g., Soller, Mones, Jermann, & Mtihlenbrock, 2005) but this is rarely 
reflected in research designs (e.g., Ilgen, Hollenbeck, Johnson, & Jundt, 2005). 

Concerning how teams process feedback, we propose that they do so by performing shared reflective 
activities, that is by collectively discussing and reflecting upon their functioning (e.g., Schippers, Den 
Hartog, & Koopman, 2007). These activities are core building blocks of team learning. Specifically, it has 
been shown that reflective teams evaluate their performance and strategies, look for alternatives to consider 
situations, and make decisions about new ways to tackle their task. The concept of team reflexivity, as 
proposed in organisational psychology, mirrors these activities (West, Garrod, & Carletta, 1997). In 
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educational settings, generic forms of intra-group reflection such as collective/social metacognition 
(McCarthy & Garavan, 2008), reflection (Edmondson, 1999), collaborative reflection (Morris & Stew, 2007; 
Yukawa, 2006), peer reflection, or reflective self-explanation (Rummel, Spada & Hauser, 2009) have been 
increasingly used. This recent research area is an extension of the work on individual reflection or reflective 
practice (e.g., Boud, Keogh, & Walker, 1985) that adds interactions and communication with peers to the 
learning process. Many authors agree that team reflexivity (in any generic sense) allows teams to reach a 
more accurate understanding of their task and, as a result, better performance (e.g., McCarthy & Garavan, 
2008; Schippers, Homan, & Van Knippenberg, 2013). Although the very recent research strand on team 
reflexivity acknowledges the importance of the dynamics of team performance when considering team 
reflexivity, the empirical work is only beginning to consider under which circumstances team reflexivity 
relates to changing performance, but not in contexts with systematic performance data on which to reflect 
(Schipper et al., 2013). Across disciplines, external and specific feedback is not systematically part of the 
reflective process while it is usually agreed that reflection can only occur if people have accurate knowledge 
about their current and desired learning state (Hattie, 2013). Also, the relation between time and timing of 
reflexivity and team performance remains in question, such as does reflecting right from the start of a team 
activity help the team get started and allows later success or does sustained reflection after events later in a 
team’s life also matter for sustained performance? 

Thus, when teams process feedback appears as a gap in both feedback and reflexivity research. 
Previous research on feedback and team performance suggests that feedback effects are not static but 
dynamic (e.g., McGrath, 1993); it cannot be understood as a single-cycle linear path from inputs (e.g., 
feedback) through outcomes (e.g., team performance). In the same vain, how teams learn from feedback 
should also be considered with a dynamic glance (Ilgen et al., 2005) 

The purpose of the present study is therefore to address the above-mentioned gaps by shedding light 
on how reflective behaviours relate to performance over a period of time in a complex, fast-paced, and high- 
workload situation in which two individuals with distributed information have to keep on learning to achieve 
success. Specifically, the following questions are explored: 1) how do teams naturally overtly reflect when 
provided with feedback depicting their performance, 2) how do reflective behaviours grow over time during 
and/or after action, 3) how does the timing at which team reflexivity occurs relate to performance? 


1.1 Feedback interventions 

Prior to addressing feedback in team settings, it is critical to briefly solicit input from multiple 
disciplines to better understand how the much more substantial body of research on feedback given to 
individuals have shaped the feedback concept in teams. 

In the learning sciences, feedback is an instructional practice that is expected to enhance motivation 
and learning (Mulder & Ellinger, 2013; Shute, 2008). Learning scientists have acknowledged feedback as a 
key characteristic of quality teaching decades ago in non-team settings (e.g., Mory, 2003; Shute, 2008, Yang 
& Carless, 2013). Much of the extensive work on feedback given to individuals has come to two main 
conclusions: 1) learners should be given feedback containing learning information (e.g., Duijnhouwer, Prins, 
& Stokking, 2012; Gibbs & Simpson, 2004) and 2) researchers should consider feedback from the 
perspective of the feedback receiver and thus incorporate the uptake and the receptivity of feedback in the 
feedback process (e.g., Boud & Molloy, 2013; Eva et al., 2012). This also introduces the idea of a feedback 
process that goes beyond the provision of feedback (Mulder, 2013). Since feedback is traditionally part of 
instructional programs, the drawback of multi-component interventions is that it is not always possible to 
assign behavioural changes to feedback interventions (e.g., van der Pol, van den Berg, Admiraal, & Simons, 
2008). Further, most studies concern primary school and high school students (e.g., Johnson & Johnson, 
1993), which raises the question of the generalizability of findings to higher education or workplace (adult) 
learning. 

By contrast, organisational, social, and behavioural psychology have incorporated feedback delivery 
in many (semi)experimental research and extensively investigated its added value with or without other 
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components (such as goal setting) to human performance (e.g., Kluger & DeNisi, 1996) while the feedback 
process has largely remained a black box. Furthermore, feedback is often mere “knowledge of performance 
or results” (e.g., performance data of a company or score on a simulation game) instead of elaborated 
informational feedback (e.g., Austin, Kessler, Riccobono, & Bailey, 1996). The learning value of feedback 
seems to be a consistent omission. Moreover, this research tradition has primarily focused on post-secondary 
levels. 


Taken together these disciplines have given rise to a new question transcending the simple question 
of whether or not feedback is truly effective: how and under which conditions feedback improves learning 
and performance. This concern has been echoed in the relatively smaller research strand on feedback to 
teams. 


1.2 Feedback to teams 

Feedback at the team level of analysis is defined as the communication of information provided by 
(an) external agent(s) concerning actions, events, processes, or behaviours relative to task completion or 
teamwork (Gabelica et al., 2012, London, 2003). 

It is widely accepted that feedback can provide teams with accurate information on their 
performance and may steer, motivate, support, and reinforce future team behaviour. Feedback is considered 
as a leverage point in the team's development of a collective view of expectations and awareness about its 
behaviours, capabilities, and skills (London & Sessa, 2006; Prins, Sluijsmans, & Kirschner, 2006). Research 
in collaborative learning environments has highlighted that feedback has the power to draw the team’s 
attention to specific aspects of its task and hence encourage task-related discussion (Johnson & Johnson, 
2002). In the workplace, feedback can also serve as a motivational trigger. For example, Scott-Young and 
Samson (2009) showed that providing teams of managers with performance feedback reinforced teams’ 
confidence in themselves and in turn, their performance. 

Despite many potential benefits of feedback delivery, a recent review by Gabelica et al. (2012) 
integrated findings from fifty-nine empirical studies investigating the effects of feedback in teams in 
educational and professional settings and showed mixed results. Approximately one third of the studies did 
not find support for its expected positive effect on performance. For example, in a field experiment, Jung and 
Sosik (2003) found that giving feedback to teams performing decision-making tasks had positive benefits on 
group members’ collective confidence (i.e., collective efficacy and group potency) but not on team 
performance. Based on these inconsistent results, analogue to feedback research in non-team settings, 
Gabelica and colleagues (2012) concluded that the key question of whether team feedback is effective 
depends on the conditions under which feedback is given, and not only on feedback as such (e.g., its quality). 

Based on educational research, it can be argued that in addition to factors related to the feedback 
giver and environment, feedback receivers have a critical role to play. Research on team feedback suggests 
that teams given feedback will only change if they perceive a learning need and opportunity and if they 
attend, interpret, and act upon feedback (e.g. London & Sessa, 2006; Phielix et al., 2010). In other words, 
teams need to proactively process the content of feedback, and thus invest time and effort into actively 
building content-oriented reactions if we expect visible changes in the way they perform. Yet, how teams 
process information cues contained in feedback, and thus what specific processing behaviours and activities 
are dynamically related to performance remains largely unknown (Gabelica et al., 2012). Although there are 
few studies on peer feedback exploring the role of feedback receivers during the feedback process in teams 
(e.g. Prins et al., 2006), the interconnections between uptakes of feedback receivers and ongoing 
performance are still unclear. Also, since the success of feedback in terms of an effective uptake from the 
receivers depends at least partially on the feedback quality provided by others, studying the uptake of 
standardised feedback (i.e., of constant quality and constant source) would allow us to isolate the learning 
effects of providing feedback. 

In sum, there seems to be an agreement that reaching an inter subjective understanding of the content 
of the feedback in teams by discussing what can be learned and worked out from past experiences is a potent 
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factor augmenting feedback effectiveness (e.g., Boud et al., 1985). Despite a lack of direct evidence 
establishing the benefits of feedback processing behaviours, the consensus appears to be that the construct 
holds enough potential to warrant further investigation. The recent research strand on team reflexivity 
depicting the extent to which teams reflect upon and modify their functioning informs our understanding of 
these processing behaviours (Schippers et al., 2013). 


1.3 Team reflexivity 

In pedagogy, individual reflection- or reflective practice- can be traced back to the early 1900s 
(Dewey, 1910, 1997) but has been introduced more extensively into the field of professional learning by 
Schon (1983) as professionals’ critical consideration of what they are working on while they are working on 
it. On a simple level, one can consider reflection in the past, present, and future tense. Schon refers to 
6 reflection-in-action’ as analysis in the present tense (i.e., reflection on the spot) and ‘reflection-on-action’ as 
analysis in the past tense (i.e., review of past actions). Killion and Todnem (1991) underlie a lack of forward 
thinking implicit in Dewey’s work and propose that reflection should also inform future action. Thus, they 
added ‘reflection-for-action’ as reflection oriented towards the future (i.e., identification of guidelines to 
follow to succeed in the future). 

Reflection as an individual critical thinking process has been recently extended to a view of 
reflection as a collaborative critical thinking process consisting of cognitive and social interactions between 
two or more individuals who examine their experiences to construct novel inter subjective understandings 
(Boud et al., 1985; Yukawa, 2006). As such, it is considered as a core team learning process. Work on team 
learning has demonstrated that collective learning can be realised through iterative sequences of action, 
reflection, and implementation (Dochy et al., 2014; Edmondson, 1999). In the learning sciences, there are 
multiple labels denoting this concept of reflection at the team level. For example, the following terms have 
been used: collaborative reflection (Morris & Stew, 2007; Yukawa, 2006), peer reflection, reflective self¬ 
explanation (Rummel, Spada, & Hauser, 2009), or collective or social metacognition (McCarthy & Garavan, 
2008). In small group research, principally one label “team reflexivity” has been introduced by West (1996) 
as a set of collaborative reflective behaviours and activities during which the team objectives, strategies, and 
processes are discussed openly. We use the term “team reflexivity” throughout this paper as a unique label 
for reflection at the team level. 

Originally, the concept of team reflexivity has not been explicitly connected to the feedback process. 
However, it is generally acknowledged that reflection is enabled by feedback to ensure accuracy in learning 
(Hattie, 2013). As a result, team reflexivity can be conceptualised as ways teams collectively try to extract 
meaning and cues for future behaviours from received feedback, generate intentions and plans, and 
ultimately decide to act upon feedback. Thus, when performance feedback that is merely evaluative is given 
to teams, the process that follows this feedback moment might be shared reflection on the task and the team 
process. The underlying assumption is that team feedback gives goal-oriented information but teams are still 
responsible for its mindful uptake. It can be argued that reflective teams consider reasons, rationales, and 
evidence for this evaluation of past performance, weigh alternative perspectives to construct a better 
understanding of their collective experience that, in turn, better guides their future action (Yukawa, 2006). 

Three behaviours that reflect complementary dimensions of team reflexivity can hence be derived 
from previous work on team reflexivity across disciplines (e.g., Schippers et al., 2007; Yukawa, 2006): (a) 
evaluating present and past performance and strategies, (b) looking for alternatives, and (c) making 
decisions. Evaluating refers to team members reviewing their goals, performance, strategies, and possible 
reasons behind success or failures. Looking for alternatives occurs when teams make an inventory of 
possible ways to achieve the task. Finally, making decisions consists of clearly stating a decision about how 
to handle the task differently and acting upon it. Evaluating and looking for alternatives reflect the capability 
of the team to be self-aware of its behaviours and the necessity to make changes. According to Schippers and 
colleagues (2007), this is necessary but not sufficient to engage in change. Teams also need to implement the 
adapted actions. This is reflected by our conceptualisation of ‘’making decisions” that depicts both the 
‘’intention to act” and ‘’carrying out the decision”. Hence, this suggests a time-ordered sequence of 


68 I F L R 


C. Gabelica 


£ 

reflective behaviours that might constitute reflective cycles, although no empirical work supports the 
necessity of full three-phase sequences. 

Overall, reflective teams have the ability to uncover why they succeeded or failed, solve 
misunderstandings, and correct their future approaches as new challenges emerge (Tschan, Semmer, Nagele, 
& Gurtner, 2000; Wills & Clerkin, 2009). As a consequence, team reflexivity has been recognised as an 
important contributor to effective collaboration and performance (e.g., Kramarski, 2004; Rummel, Mullins, 
& Spada, 2012; Schippers, Den Hartog, Koopman, & van Knippenberg, 2008; Tjosvold, Tang, &West, 2004; 
van Ginkel, Tindale, & Knippenberg, 2009). However, in their review of small group research Moreland and 
McMinn (2010) draw attention to 1) the lack of significant relation between reflexivity and team 
performance found in some studies (e.g., Edmondson, Bohmer, & Pisano, 2001; Savelsbergh, van der 
Heijden, & Poell, 2009) and 2) relatively limited evidence of the effect of reflexivity on team performance 
(e.g., Lewis, Belliveau, Herndon, & Keller, 2007; Muller, Herbig, & Petrovic, 2009). They concluded that 
reflexivity could be beneficial to team performance under certain circumstances. In the learning sciences, a 
similar trend has been observed: although team reflexivity in collaborative teams is highly important for the 
learning process, it does not always yield better learning gains (e.g., Prinsen, Terwelb, Zijlstrac, & Volman, 
2013). 

Given these mixed results, limitations of the small but growing research strand on team reflexivity 
need to be synthesised. First, reflexivity does not happen in a vacuum. Teams will eventually adapt their 
operating methods and ways of working based on feedback cues from their environment. We could expect 
reflexivity to only improve team performance when teams have access to feedback describing their objective 
and accurate performance (Schippers et al, 2013). Yet, reflexivity is seldom conceptualised as a process 
augmenting the effect of feedback on performance (Seibert, 1999). Moreover, little is known about how 
people reflect on feedback at the team level, while there has been empirical evidence of the effect of 
reflection upon feedback at the individual level (e.g., Duijnhouwer et al., 2012). For example, Anseel, 
Lievens, and Schollaert (2009) have tested the effect of feedback augmented with reflection at the individual 
level and demonstrated that the combined use of individual-level feedback and reflection improved 
performance better than individual feedback alone. At the team-level, only one series of studies isolated the 
effect of feedback from its combination with reflection in computer-supported collaborative learning in high- 
school teams (Phielix, Prins, & Kirschner, 2010; Phielix, Prins, Kirschner, Erkens, & Jaspers, 2011). The 
authors expected shared self and peer assessment and shared reflection to have complementary effects. They 
did not find any significant effect of reflection alone or of the combined use of feedback and reflection on 
objective performance (i.e., grade), but demonstrated that the joint use of feedback and reflection lead to 
higher group process satisfaction and social and cognitive behaviour. Interestingly, they draw attention to the 
fact that feedback (based on peer and self-perceptions) and reflection did not decrease unrealistic positive 
perceptions teams generally have about their own and other performance. This could be a reason for a lack of 
effect on objective performance. We do not know what are the effects of external feedback based on 
objective criteria for task achievement. 

Second, as in most research in organisational psychology, the vast majority of small group research 
measuring team reflexivity in relation to team performance has used self-report instruments. Self-report 
measures are limited by team members’ level of awareness of their own behaviours and states and distorting 
biases such as social desirability. Calls for studying reflective behaviours in teams have generally gone 
unheeded (West, 1996). In the learning sciences, Dillenbourg, Baker, Blaye, and O’Malley (1996) have 
advised researchers to zoom in the ‘black box’ of collaborative processes. Subsequent to this call, there has 
been a recent proliferation of process-oriented research on collaboration (e.g., De Wever, Schellens, Valcke, 
& Van Keer, 2005) that advanced our understanding on interaction features contributing to more effective 
learning. Most of this collaborative learning research strand has focused on individual learning without 
explicitly investigating how collaborative processes influence team performance (e.g., Janssen, Kirschner, 
Erkens, Kirschner, & Paas, 2010). Nevertheless, these insights underscore the value of observational 
methods to provide crucial information about the context in which reflective behaviours occur and relate to 
team performance (e.g., Chi, 1997; Leicht, Hunter, Saluja, & Messner, 2010). 
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Finally, one area in which our understanding is incomplete across disciplines concerns the role of 
time and timing of reflective behaviours and their relation with team performance (e.g., Ballard, Tschan, & 
Waller, 2008; Janssen et al., 2010; Okhuysen & Waller, 2002; Reimann, 2007; Waller, 1999). There is a 
general agreement in the team literature that team performance is the product of ongoing and recurrent 
processes and actions (McGrath, 1993). Marks and colleagues (2001) conceptualise these cycles as 
performance episodes. Performance episodes consist of repeated cycles of action (i.e., when teams perform 
an activity) and transition (or ‘interrupts’) phases between actions. These interrupts are opportunities for 
teams to stop and reflect about their progresses for engaging in change (Okhuysen & Waller, 2002). The 
most common conceptualisation of team reflexivity does stipulate that shared reflection can occur before, 
during, and after a task (West, 2000; Schippers et al., 2007). However, scholars have generally measured it 
as an overall working style (Gurtner, Tschan, Semmer, & Nagele, 2007) or as an aggregated measure of 
collaborative activities and have not differentiated the moments at which it occurs (e.g., Lajoie & Lu, 2012). 
Specifically, they have not tested whether team reflexivity was more or less beneficial in certain phases, in 
relation to team performance dynamics, as suggested by certain authors (Hoeg & Parboteeah, 2006; Janssen 
et al., 2010; Schippers et al., 2003). This time-related issue of team reflexivity is elaborated upon in the 
following section. 

1.3.1 Time and timing of team reflexivity 

We identify three primary issues in understanding the dynamic aspects of team reflexivity. First, 
although the importance of dynamic conditions experienced by teams over time is widely accepted (e.g., 
Waller, 1999), empirical work on how team reflexivity changes over time is missing (e.g., Janssen et al., 
2010). On the one hand, it may be that overt communication is no longer needed as teams improve their 
implicit coordination over time, thus decreasing reflective interactions (e.g., Entin & Serfaty, 1999). 
Additionally, teams tend to define their goals and strategies at an early stage of their work and not to 
deliberatively review them after some work has been accomplished (Argote, 1989; Hackman & Wageman, 
2005). Also, in line with arguments from Schippers and colleagues (2003), reflexivity might decline over 
time in diverse teams as viewpoints and perspectives become incompatible. On the other hand, they suggest 
that this declining effect might be reduced by the provision of feedback. It may be that the availability of 
accurate performance data highlighting deficiencies and a sustained task complexity trigger learning needs 
for teams perhaps calling for more reflection over time (Rulke & Rau, 2000). As such, feedback provision 
occurring during transition can act as a formal mechanism, a temporal punctuation likely to encourage 
reflection without necessarily giving a predetermined framework to follow (Okhuysen & Waller, 2002). 

Second, the role of timing of reflective behaviours is similarly not well understood. The scarce 
previous research on the question ‘’does reflection during action and/or transition lead to better 
performance” has shown mixed results in contexts without explicit feedback. For example, in a study 
conducted by Moreland and McMinn (2010), none of the (scarce) reflective behaviours occurring during 
transition was significantly related to changes in team performance. By contrast, research looking into the 
impact of "interrupts" on group processes concluded that these were triggers of change in groups (Okhuysen 
& Eisenhardt, 2002). Team members appear to naturally interrupt their work around the midpoint of the 
allocated time for task completion and be more likely to put into practice strategies they set during these time 
outs (Gersick, 1989). Concerning reflection during action, Moreland and his colleague suggest that it might 
have more impact on performance. This reflection-in-action would be more directly related to the activities 
team members perform and thus prevent errors from being committed in real time. Conversely, it is likely 
that reflection while performing has a cost especially in a task that combines active processing of 
information and coordinated actions (Kirschner, Paas, & Kirschner, 2009; Schippers et al., 2013). That is, 
reflecting while executing a complex task places an extra burden on teams which may overload their 
working memory occasioning less optimal performance (Kirschner et al., 2009). 

Furthermore, it is generally recognised that what happens in the early part of the team interaction 
might provide insight into subsequent effectiveness (e.g., Eriksen & Dyer, 2004; Kaplan, Laport, & Waller, 
2013). Team decision-making literature has provided preliminary insights into this issue of timing of 
behaviours. It was shown that in teams with distributed information (i.e., comprising team members holding 
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unique information), early agreements might harm decision quality because teams are less focused on 
exchanging and integrating distributed information (van Ginkel et al., 2009; van Ginkel & van Knippenberg, 
2009). Accordingly, jumping too early into task completion might lead to process losses and performance 
decline (Mathieu & Rapp, 2008). On the contrary, it was demonstrated that effective teams share and 
elaborate upon distributed information at the beginning of interaction (van Ginkel at al., 2009). Rulke and 
Rau (2000) came to a similar conclusion when examining how teams develop a shared understanding of 
‘who knows what’ in the team (i.e., transactive memory systems), proved to be an important factor to 
achieve better success. They observed that teams with high transactive memory systems were those whose 
members shared understandings and evaluated each other’s expertise early in their team interactions. In 
another example from computer-supported collaborative learning research, Kapur, Voiklis, and Kinzer 
(2008) demonstrated that a high quality contribution at the beginning of a problem solving process had more 
impact than those occurring later during team interactions. Therefore, the temporal pattern within reflective 
interactions should be taken into account in further understanding team reflexivity upon feedback. Also, 
presently, in the team reflexivity literature the question ‘’do the three behaviours making up reflexivity have 
differential effects on subsequent performance at an early stage of team interaction’ ’ remains unanswered. 
No empirical work has shown the necessity of a certain order of these reflective behaviours nor whether, and 
if so when, certain reflective behaviours were more conducive to better performance. For example, if we 
look into team reflective behaviours individually, we do not know if evaluating and looking for alternatives 
during teams’ first moment of interaction promote elaboration and understanding of the task whereas making 
decisions at an early stage is detrimental to subsequent performance. 

Third, the direction of the relation between reflexivity and performance can be questioned (e.g., 
Janssen et al., 2010). In line with the core assumption of previous research on team reflexivity, does 
reflexivity lead to subsequent better performance? Alternatively or at the same time, do teams learn from 
previous performance, and thus reflect more as a consequence of how they performed previously? Research 
still has to prove the theoretical claim that teams can learn from the past through reflection with clear sight of 
performance criteria and information about their attainment. Only recently, Schippers and colleagues (2013) 
have given indirect evidence in this regard. In this study, self-report reflexivity was measured at two points 
in time in teams of students working on their bachelor thesis. This study showed that low-performing teams 
had the capability to translate information from performance feedback into effective task approaches. 
However, students were only given a grade and not feedback describing attainment of specific performance 
criteria. As suggested by Waller, Gupta, and Giambatista (2004) the timing of errors and subsequent 
behaviours has to be recorded to answer the question of causality. Further, investigating the timing of 
reflective activities in teams can help detect the points at which team reflexivity occurs and may need to be 
supported (Lajoie & Lu, 2012). 


1.4 The present study 

In the present study, we seek to understand the dynamics of team reflexivity and the relation (uni or 
bi-directional) between team reflexivity and performance. Therefore, we explore the two following 
questions. 1) Does the occurrence of team reflexivity augment or decline over time during action and 
transition phases of teamwork? 2) How is the timing at which reflective behaviour occurs related to 
performance? Specifically, is reflexivity during action and/or during transition related to higher performance 
(a)? Does each behaviour making up team reflexivity have the same impact on performance when occurring 
during teams’ first moment of interaction (b)? 


2. Method 
2.1 Participants 

Sixty-four students (32 male and 32 females) were recruited from a university in the Netherlands and 
randomly assigned to thirty-two dyads (N = 32). Their ages ranged from 18 to 29 years, M = 22.3, SD = 2.4. 
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Participants were not eligible if any of the following exclusion criteria were present: experience in flight or 
related simulations and familiarity with each other. They were either paired with a same-gender partner 
(female and male teams, n = 11 and n = 11 respectively) or different-gender partner (mixed teams, n = 10). 
By random assignment, half of the sample was assigned to a role of pilot and the other half to a role of co¬ 
pilot. Subjects participated voluntarily in exchange for vouchers. 


2.2 Task 

Participants in the role of pilot and co-pilot were required to complete four landing missions of the 
computer simulation “Microsoft Flight Simulator X”. The task, a complex, fast-paced, and high-workload 
situation, was chosen to stimulate ongoing learning in a controlled environment. Cognitively complex and 
interactive simulation tasks, such as flight-simulations, are commonly used in team research to investigate 
processes related to team performance (e.g., Bowers, Salas, Prince, & Brannick, 1992; Villado & Arthur, 
2013). We did not use this computer simulation to mimic real-work team environments but rather to examine 
a set of theoretical relations (i.e., nomological network) among constructs within specific and controlled 
boundaries: a complex, fast-paced, and high-workload situation in which team members with unequally 
distributed information have to learn from each other to achieve their team goal and extend their learning to 
more complex variations of the task (Marks, 2000). To avoid that good-performing teams would have less 
need to learn as a consequence of their reflection on performance (Schippers et al. 2013), the level of 
complexity of the missions increased gradually over time. The abundance of information teams received 
before and during the missions and the high level of interdependence between pilot and co-pilot ensured a 
high level of complexity across performance episodes. In each mission, teams had to follow a predetermined 
traffic pattern during which they were required to maintain appropriate levels of speed, altitude, and a correct 
configuration of the airplane. The missions were completed when the team managed to land safely on the 
runway. The computer was connected to a whiteboard on which the game was screened. 


2.3 Procedure 

The whole session lasted approximately two and a half hours. After introduction to the procedure 
and random assignment to the role of pilot or co-pilot, participants were individually trained during forty- 
five minutes. Items of information necessary for achieving a good landing were distributed between the team 
members. Pilots and co-pilots were seated in separate rooms to study the task material containing critical 
role-specific knowledge of piloting or monitoring the aircraft. The task of the pilot was to fly the plane and 
operate the joystick. For that purpose, pilots received an additional 10-minute hands-on training to practice. 
The task of the co-pilot was to control the gas of the plane and provide the pilot with indications and 
directions. Only the co-pilot had the access to the air traffic control (ATC)’s instructions, given through 
headphones, and knew how to interpret the cockpit instruments. After the training, participants were seated 
together to complete four landing missions. Teams had up to fifteen minutes to complete each mission and 
were also allowed to restart a mission if they had crashed. Before each mission the team received a written 
description of the flight objectives and the general mission scenario. Moreover, before starting missions 2, 3, 
and 4, teams were given specific performance feedback about their previous performance. Performance 
feedback described the attainment of success criteria such as speed, altitude, rate of descent, pitch, 
touchdown, and traffic pattern. The participants were allowed to communicate freely with one another. All 
teams were videotaped. 


2.4 Measures 

2.4.1 Team performance 

Two performance scores were computed: the total number of errors during a mission and the number 
of times teams crashed. The number of errors was derived from an instrument rating objective performance 
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criteria (e.g., speed, altitude, activation of flaps and landing gear, landing position) of a good landing 
approach. This instrument was based on two sources: firstly, to identify key factors of a good flight, we 
performed a task analysis with a flight expert. Secondly, we used tests that the game itself provides its 
players to refine these criteria. Examples of deficiencies (i.e., errors) included failure to extend the flaps 
before landing, to maintain a certain speed interval during descent, to reduce the speed before touchdown, to 
keep a constant rate of descent, to align with the runaway, or to have one touchdown on the runaway. The 
total number of potential errors varied in the four missions. This variation reflects the increasing level of 
difficulty of the missions. We chose the number of crashes to depict one of the most salient manifests of 
performance for participants. There were four measurement times in total (i.e., Tl, T2, T3, and T4). 


2.4.2 Categories of team reflective behaviours 

Team communication was coded to identify representative behaviours that could be taken as 
evidence of team reflexivity (Rourke & Anderson, 2004). We developed the coding scheme of the present 
study through a series of steps assuring its validity and reliability (Schippers et al., 2007). First, we 
determined the granularity of the unit of analysis. The unit of meaning was applied (Rourke, Anderson, 
Garrison, & Archer, 2001). Specifically, to consider a verbal statement a significant unit, we decided that 
utterances had to be individual messages (questions or statements) that 1) were expressed by one team 
member, 2) dealt with one topic, idea, or argument chain, 3) reflected one unique behaviour, and 4) related to 
the topic at hand or the team. Thus, one semantic feature (unit of meaning) and one activity feature (team 
member speaking) were used for segmentation of the communication content into units (Chi, 1997). As such, 
as soon as the topic or the speaker changed, a new behaviour was coded (Visschers-Pleijers, Dolmans, de 
Leng, Wolfhagen, & van der Vleuten, 2006). In addition to verbal statements, one unambiguous non-verbal 
behaviour was set as an evidence of one of the reflective behaviours. 

Second, we discriminated verbal interactions types that typified reflexivity. To do this, we adapted 
and expanded the initial framework of team reflexivity (West, 1996, 2000) and an existing questionnaire 
from Schippers and colleagues (2003). Reflexivity was originally defined as an iterative process including 
three broader behaviours, namely reflection, planning, and acting/adapting. As shown in Table 1, the coding 
scheme covers three reflective behaviours: evaluating or reviewing present or past team performance and 
strategies, looking for alternatives, and making decisions. Information directly forwarded from the ATC 
(repetitions) and the literal reading of the feedback form were excluded from the coding. 

Table 1 


The Coding Scheme for the Content Analysis of Team Reflexivity 


Categories _ 

Evaluating or reviewing 
performance or strategies 


Description _ 

Statements or questions about 
team performance (e.g., whether 
the team does/did well, is/was on 
the right track according to plans 
or received instructions), the goal 
of the mission and its 
requirements, actions and 
strategies (mis)used, reasons 
behind success, failure, or 
problems (e.g., he/she gives 
examples of behaviours, task or 
team strategies that may explain 
why they achieved success or 
encountered problems during this 
mission). 


Examples _ 

“We are going in the wrong 
direction.” 

“We crashed because we were 
always too fast.” 

“Something went wrong, maybe 
the nose of the plane went too 
low?” 
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Looking for alternatives 


Making decisions 


Suggestions or discussions of 
alternatives in how they 
approached the task (at the task or 
team levels) and of the sequence 
of actions undertaken. In other 
words, teams discuss how they 
could do or could have done 
differently. 

Statements clearly depicting a 
decision about a new direction to 
take or observable behaviours 
following a decision. Team 
members’ utterances depicting 
very explicit decisions about the 
way they were going to approach 
the task or work as a team, explicit 
statements about the intention to 
follow decisions made within the 
team, and explicit reaction to a 
decision by an action (e.g., by 
pressing the flaps, pulling the gas 
controller). 


“We could have reduced the 
speed by pitching up or reducing 
the throttle.” 

“We could lower the speed by 
extending the flaps, pitching up, 
or lower the gas.” 

“We could either make a U-turn 
either still try to lower speed and 
make a sharp descent.” 

6 ’We are going to make a U- 
Turn” or “This time, you look at 
the speed indicator and I will 
pitch down”. 


Third, we ran a pilot study to test and validate the coding scheme. This lead to adaptations, 
clarifications of the reasoning behind the framework definitions and the boundaries of the units, and the 
addition of typical examples. Fourth, we extensively trained two coders, each blind to the hypotheses of the 
study, to optimise reliability and consequently reduce errors in observation. They were provided with clear 
examples (of inclusion and exclusion) of the manifestation of the behaviours and had rating exercises with 
multiple rounds and discussions to attain consistency among coders. Fifth, videotapes were coded with the 
newly developed coding scheme. Finally, the two coders coded independently one-third of the videotapes to 
estimate interrater reliability (Cohen’s kappa). Kappas were calculated for all the categories. These kappas 
ranged from 0.65 to 0.88, with an average of 0.78, indicating a ‘substantial’ to ‘almost perfect agreement’ 
across the two coders as to the occurrence of the specific behaviours (Landis & Koch, 1977). Coders and two 
trainers resolved any discrepancies. 

The research design is displayed in Figure 1. 
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Figure 1 . Overview of the phases of the study and behaviours measured at each wave of data. 


3. Analyses 

For the coding, we used The Observer® XT 10.5, a computer software aimed for quantitative 
analysis of observational data. Videotapes were directly coded without transcripts. The extent to which teams 
engaged in each behaviour was expressed in terms of frequencies of occurrence of the behaviour of its 
members for each mission (i.e., action phase) and for each feedback (transition) phase. Additionally, we 
computed an overall team reflexivity score (i.e., aggregation of the three behaviours) for each time 
measurement for action and feedback. 
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We coded acts on the basis of utterances of reflective behaviours occurring at the team level of 
analysis (i.e., aggregation of individual utterances for each team). Besides utterances, we examined phases 
(or ‘’when”, specifically action or transition, earlier interaction or later interaction) in which some 
performance events (i.e., crashes and errors) were related to team reflective behaviours. Behaviours were 
coded at four points in time during action and feedback phases (Time 1, Time 2, Time 3, and Time 4) for 
each team (N = 32). 


4. Results 

In the following section, we first present an overview of the frequencies of behaviours considered 
individually and of the frequencies of sequences comprising two or three behaviours. Second, we test 
whether reflective behaviours change over time during action and feedback using repeated-measures 
analyses of variance. Third, correlations between team reflective behaviours and performance are examined, 
more specifically (a) the relations between prior performance and subsequent team reflexivity and (b) prior 
team reflexivity and subsequent performance. 


4.1 Frequencies of behaviours at the team level of analysis 

Figures 2, 3, and 4 depict means and standard deviations of the reflective categories across time 
during action and feedback phases. It can be seen that evaluating is the most frequent reflective behaviour. 
Looking for alternatives during action at Time 1 appears very scarce. During feedback, it seems that looking 
for alternatives is not a frequent practice. The same trend can be noted for making decisions. It is more 
frequent during missions than after feedback reception. It has to be noted that standard deviations reflect 
important differences between teams. In sum, reflective behaviours, when examined individually, tend to 
follow a similar pattern: they appear more frequent during action while they are low during feedback. 

14 
12 
10 
8 
6 
4 
2 
0 

Figure 2. Means and standard deviations of Evaluating for each measurement time and phase (N = 32). 
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Figure 3. Means and standard deviations of Looking for alternatives for each measurement time and phase 
(N = 32). 
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Figure 4. Means and standard deviations of making decisions for each measurement time and phase (N = 
32). 


4.2 Sequences of evaluative behaviours 

The coded reflective acts described above are single communication behaviours. In the present 
study, team reflexivity was conceptualised as a collection of three behaviours. We explored whether teams 
actually completed full ‘’reflective cycles” comprising all three behaviours in a sequence. Since there has 
been no empirical work demonstrating the necessity of all three behaviours we also considered the most 
basic behavioural patterns consisting of two subsequent reflective behaviours. As can be seen in Table 2, 
most reflective communication across teams can be summarised by two main two-behaviour sequences: 
sequences starting with evaluating and looking for alternatives and ending with clear decisions about a 
different way to handle the task. In these sequences, teams “skipped” the evaluation or search for alternatives 
phases. While sequences of looking for alternatives followed by decisions seemed to grow over time during 
action, except for the last mission, the frequencies of the sequences starting with an evaluative comment and 
ending with a decision stayed relatively stable over time, except for a drop at Time 2. During feedback, the 
same trend than for individual behaviours is observed; sequences are very scarce. Importantly, full cycles 
were almost never completed, suggesting that teams were not naturally systematic in their reflective process. 
The absence of reflective cycles does not allow us to further investigate their change over time and 
relatedness to team performance. Still, how individual reflective behaviours evolve over time and are related 
to team performance is of importance to map 1) how teams naturally respond to feedback and 2) whether 
some behaviours appear more important than others in the feedback process. 


77 I F L R 




















































C. Gabelica 


& 

Table 2 

Frequencies of Sequences of Reflective Behaviours at each Wave of Data (N - 32) 


Sequences 


Time 1 


Time 2 


Time 3 

Time 4 


Action 

Feedback 

Action 

Feedback 

Action 

Feedback 

Action 

B1-B2 

2 

2 

3 

0 

15 

1 

9 

B1-B3 

30 

14 

15 

7 

29 

0 

29 

B2-B3 

2 

2 

18 

0 

30 

3 

23 

B1-B2-B3 

0 

0 

1 

0 

2 

0 

0 


Notes. B1 = Evaluating, B2 = Looking for alternatives, B3 = Making decisions. 


4.3 Does team reflexivity change over time? 

To test for significant changes of team reflexivity behaviours (considered individually) over time, we 
computed repeated-measures analyses of variance with a Greenhouse-Geisser correction (as sphericity was 
violated for all behaviours) and with the four times each behaviour was measured as a within-team factor. 
Pairwise comparisons with Bonferroni corrections controlling for inflation of Type I error were also 
computed. Evaluating during action changed significantly from Time 1 to Time 4, F (2.26, 65.67) = 5.04, p = 
.05, with pairwise comparisons showing that evaluating at Time 4 was significantly more frequent than 
evaluating at Time 2 (p = .019). In contrast, evaluating during feedback did not change over time. Regarding 
looking for alternatives during action there was also an overall significant difference between the means at 
the different time points, F (2.24, 64.84) = 5.76, p = .004. A pairwise comparison confirmed a difference 
between Time 1 and Time 2 (p = .034), Time 1 and Time 3 (p = .000), and Time 1 and Time 4 (p = .0006), 
signifying an increase of the behaviour over missions. Looking for alternatives during feedback, making 
decisions during action and during feedback did not change significantly over time with a Bonferroni 
correction. With Tukey’s test, making decisions during action at Time 4 was significantly higher than at 
Time 3 (p = .043) and Time 2 (p = .045). Finally, while overall reflexivity across feedback did not change 
significantly over time, the mean scores for overall reflexivity across missions (i.e., aggregated behaviours) 
were significantly different, F (2.33, 67.56) = 5.46 p = .004. Specifically, reflexivity during mission at Time 
4 was higher than reflexivity during mission at Time 1 (p = .027) and Time 2 (p = .035). It is worthwhile 
noting that if the less conservative Tukey post hoc test is used, the score in mission 3 is higher than at Time 1 
(p = .015) and Time 2 (p = .043). 


4.4 Is team reflexivity related to team performance? 

Intercorrelations between reflective behaviours and performance measures are presented per time 
period in Table 3. 
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Table 3 

Correlations between Coded Categories and Performance Measures at each Point in Time (N - 32) 


Reflective 

Errors 

Crash 

Errors 

Crash 

Errors 

Crash 

Errors 

Crash 

behaviors 

mission 

mission 

mission 

mission 

mission 

mission 

mission 

mission 


1 

1 

2 

2 

3 

3 

4 

4 

Evaluating 
Mission T1 

4 ^ * ^ 








Evaluating 
Mission T2 

.45* 


.43* 






Evaluating 
Mission T3 

.50* 








Evaluating 
Mission T4 




-.42* 





Evaluating 
Feedback T1 









Evaluating 
Feedback T2 




-.45* 

-.42* 




Evaluating 
Feedback T3 








-.45* 

Alternatives 

Mission T1 









Alternatives 

Mission T2 



.61** 






Alternatives 

Mission T3 

.36* 

.38* 







Alternatives 

Mission T4 









Alternatives 

Feedback T1 









Alternatives 

Feedback T2 

.51** 








Alternatives 

Feedback T3 









Decisions 

Mission T1 

4 ^ * ^ 





.48** 



Decisions 

Mission T2 

.51** 

. 47 ** 

.51** 






Decisions 

Mission T3 

.48** 








Decisions 

Mission T4 



-.42* 


-.43* 




Decisions 

Feedback T1 

.45* 

.39* 




.38* 



Decisions 

feedback T2 

.43* 


.43* 






Decisions 

feedback T3 
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Note. *p < .05. ** p < .01. Tl= Time 1; T2=Mission 2; T3=Mission3; T4=Mission 4. Only significant 
correlations are indicated 

4.4.1 The impact of prior team performance on team reflexivity 

At first glance, initial errors seem to be beneficial to subsequent reflective behaviours, while the 
trend that errors trigger reflection tends to decline with time. Specifically, the number of errors teams made 
during the first action phase was positively correlated with numerous reflective behaviours and to all 
reflective categories. The number of crashes teams initially faced followed the same path. Teams did not 
seem to be discouraged by their first experience with crash. For example, the initial number of crashes was 
positively related to more decision-making behaviours after it occurred, both during the feedback phase 
following the “failure” (r = .39, p < .05) and during the next action at Time 2 (r = .47, p < .01). In contrast, 
errors committed at Times 2 and 3 were not related to higher subsequent reflective behaviours, with the 
exception of errors at Time 2 appearing as a trigger for decision making during the transition phase 
immediately following that mission (r = .43, p < .05). Conversely, errors at Times 2 and 3 seem to hamper 
decision making during action at Time 4 (respectively r = - .42, p < .05 and r = - .43, p < .05), while the 
number of crashes at Time 2 was related to less frequent evaluative behaviours during feedback at Time 2 (r 
= -.45, p < .05) and during action at Time 4 (r = - .42, p < .05). In sum, these first results seem to suggest that 
initial failure acts as an eye opener to evaluate what went wrong, look for alternative ways of approaching 
the task, and make more decisions, while later, as task becomes more complex, it might relate to less 
subsequent reflection. 

4.4.2 The impact of team reflexivity on subsequent team performance 

Concerning the impact of team reflexivity on subsequent performance, the correlations indicate a 
differential impact depending on the type of behaviour considered. The extent to which teams evaluated their 
performance and strategies during feedback at Time 3 was related to lower number of crashes at Time 4 (r = 
-.45, p < .05). Similarly, the extent to which teams engaged in evaluative behaviours during feedback at 
Time 2 significantly related to fewer errors at Time 3 (r = -.42, p < .05). What is particularly noticeable is 
that these significant relationships concern 1) evaluative behaviours and 2) periods of team transition, 
suggesting a positive effect of processing feedback on subsequent performance. Reflection during action 
does not seem to be significantly related to subsequent performance when bivariate correlations are 
computed. 

In contrast, the extent to which teams made decisions during action at Time 1 was related to more 
crashes at Time 3 (r = .48 p < .01), suggesting that making decisions during the initial moments of team 
development on this novel task (with its specific characteristics) may impede subsequent performance. 


5. Discussion 

To uncover when giving teams feedback about their performance creates an opportunity for learning, 
it has been posited that it is important to examine how teams actively process feedback and thus 
collaboratively evaluate information about past activities and derive alternative recommendations for next 
action. However, research on this feedback processing has been scarce in the learning sciences and 
organisational psychology (Gabelica et al., 2012; London & Sessa, 2006; Phielix, et al., 2010). It is shown 
that the specific activities teams perform to deal with feedback and when these activities are related to 
performance remain to be considered. 

Following from these perceived gaps, we conducted a study attempting to build upon and extend 
research in this domain by 1) identifying actual behaviours enabling feedback processing (i.e., team 
reflexivity) and 2) providing a more fine-grained analysis of dynamic aspects of team reflexivity in a context 
with systematic and explicit feedback. While theoretical work seems to suggest team reflexivity is an 
iterative three-step cycle that involves evaluating performance and strategies, looking for alternatives, and 


80 I F L R 


C. Gabelica 


£ 

making decisions (e.g., Schippers et al., 2007; Yukawa, 2006), we do not know if to perform well, it is 
necessary to follow this series of steps and if some steps are more dominant and influential than others in 
relation to team performance. We explored the development of team reflective behaviours and the relation 
between timing of reflective behaviours and performance during four performance episodes. 

Following conclusions could be drawn. Firstly, teams never completed full cycles of evaluating, 
looking for alternatives, and making decisions. They did, however, completed two behaviour-sequences 
starting with evaluating or looking for alternatives and ending with a clear decision. Moreover, these 
sequences were very scarce during transition (i.e., interrupts during which team performance feedback was 
delivered). While the reflective cycle is usually described sequentially, teams seem to rarely follow a rigid 
series of steps to deal with feedback. Instead, it seems like they often skip steps or even go back through 
steps several times. 

When taken individually, team reflective behaviours were overall more frequent and increased over 
time during action, whereas team reflexivity during transition was relatively less frequent and did not 
change. Looking for alternatives was very scarce but also increased over time during action only. There 
might be two reasons for this growth of reflection-in-action. First, natural reflection might arise as a response 
to an immediate learning need when a team observes cues of ineffective behaviours or experience 
misunderstanding or uncertainty (e.g., they get lost) while completing the task. Additionally, this learning 
need could have been triggered by the increasing complexity of the task (i.e., more cues to understand and 
interpret). Second, preceding feedback, which has been advanced as a way to counteract the natural decline 
of team reflexivity (Schippers et al., 2003), could also have had a delayed effect on next reflection-during 
action. In the learning sciences, feedback has been shown to impact subsequent learning (e.g., Mory, 2003). 
It may be that teams only see the learning content of a feedback when they have to deal with a similar 
situation. This high reflexivity-in-action suggests that teams are more reactive than proactive and adaptive to 
anticipated circumstances when faced with higher complexity and workload. This is in line with a common 
rationale some authors have been using to speculate on the causes of a lack of actual reflection in teams 
(Arvaja, Hakkinen, Etelapelto, & Rasku-Puttonen, 2000; Morris & Stew, 2007). They state that teams are 
more driven by their results (i.e., producing and performing) than the learning they can gain, especially when 
they are under pressure and despite the obvious benefits of strategy development (Gurtner et al., 2007; Karau 
& Kelly, 1992). A possible explanation of why teams did not use transition phases to increase reflection in 
the same way lies in teams’ tendency to set their goals and strategies early and not to actively question them 
later (Argote, 1989; Hackman & Wageman, 2005; Weingart, 1992). Further, it may be that the concurrent 
cognitive demands of collaborating on the complex task, making sense of the received feedback, and trying 
to prepare for the next task were too high for teams to make the most of their learning experience (Kirschner 
et al., 2009; Rummel & Spada, 2009). Reflecting is a challenging high-order activity (Jay & Johnson, 2001). 
The concept of reflection assumes individuals have the capacity to engage in self-examination and open- 
minded analysis of their own knowledge. Additionally, even if team members can reflect in solo-learning 
situations, they may not be able to coordinate and co-reflect in a team, communicate their reflective 
thoughts, nor agree on ways to address the task (Chan, 2012). 

Secondly, we uncovered patterns that transcended the straightforward question of whether team 
reflexivity can change performance in teams given explicit feedback. It seems that the key question is rather 
when improvements occur. (Lajoie & Lu, 2012) As signified in recent research on team reflexivity (e.g., 
Moreland & MacMinn, 2010; Schippers et al, 2013), reflexivity was not uniformly beneficial. We showed 
instances in which timing of reflective behaviours determined its effect (positive, negative, or neutral) on 
performance. First, early decision-making was related to lower subsequent outcomes. These findings are in 
accordance with previous studies on timing of decision-making indicating that early decision making might 
prevent deep processing and sharing of unevenly distributed information (van Ginkel, et al., 2009; van 
Ginkel & van Knippenberg, 2008). Second, after that first experience with their task, teams were able to 
derive insights from past performance (depicted by feedback) and correct misunderstandings that prevented 
effective action (Tjosvold et al., 2004). However, only evaluative behaviours performed during feedback 
phases were related to later improved performance. For example, teams were able to reduce their errors in 
the last high-workload task with more preparation (i.e., evaluating during preceding feedback time). As such, 
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the effect of evaluative behaviours seems to be contingent on the phase during which they are performed 
(during transition rather than during action). This points a paradox: though they could experience positive 
consequences of reflecting during these time outs, teams did not increase reflection during feedback over 
time. Third, reflection during action does not seem to be significantly related to better subsequent 
performance when bivariate correlations are computed. As such, no empirical support could be found for 
Moreland and McMinn’s (2010) proposition that reflecting during the task could be more beneficial to teams 
due to the immediate possibility to adjust to the situation. However, this (increasing) reflection during action 
did not harm team performance either and showed that, in general, teams remained connected despite the 
increasingly higher workload of their task. Communication breakdowns could have been expected. Previous 
team research on non-routine working situations (e.g., Waller et al., 2004) has demonstrated that low- 
performing teams become cognitively overwhelmed in case of high workload and consequently tend to focus 
more on their individual sub-task instead of collaborating (e.g., Salas, Rosen, & King, 2007). 

Finally, initial errors appeared to be a driver of subsequent team reflexivity while later errors mostly 
did not have this motivational role. These preliminary findings open up the possibility that there might be 
some time-specific effects of previous errors that are determinant to trigger motivation to improve. Maybe 
the first mistake does not really hurt while repeated errors would be more detrimental to performance? 

Importantly, these results raise the question of the causality of the relation between team reflexivity 
and performance (e.g., Janssen et al., 2010). These analyses seem to rather show a more dynamic and 
retroactive relation between past and future performance and team reflexivity, raising the need for time- 
series-analyses. This is however behind the scope of the present explorative study. This suggests that the 
question “does prior performance trigger team reflexivity or does prior team reflexivity generate better 
performance” should rather be changed into: when do performance and team reflexivity dynamically interact 
to trigger team learning and better subsequent performance. As we found a reversed effect as well, it could 
be formulated that initial errors do matter and that teams have the capability to learn from them under certain 
circumstances. 

Taken together, these findings underscore the importance of a careful evaluation of how the team is 
doing and why during transition phases corresponding to feedback reception. We could not empirically test 
the effects of three-step reflective cycles since they did not naturally occur but we provided evidence that 
teams using the feedback opportunity to stop and analyse their performance and strategies were able to 
translate information about performance into corrective behaviours since their performance got improved. 
These results are line with studies on the impact of interrupts (e.g., Okhuysen & Waller, 2002) and theories 
on feedback in teams that stipulate that feedback receivers’ involvement plays a critical role to explain 
feedback effectiveness in teams (e.g., Gabelica et al., 2012). 


6. Limitations and future directions 

Although we have specified behaviours signifying team reflexivity, we have not explored depth of 
processing (Volet, Summers, & Thurman, 2009). It is likely that deeper reflection (e.g., reflective statements 
conveying inferences) generates better insight into the feedback content and use in subsequent tasks (Anseel 
et al., 2009). 

Also, previous research has acknowledged that developing strategies for action is important and 
necessary but does not always ensure actual strategy implementation (Gurtner et al., 2007; Marks et al., 
2001; Tschan et al., 2000). Our third reflective category (i.e., making decisions) encompassed both strategy 
development (i.e., clear decisions about new strategies and actions) and strategy implementation (i.e., clear 
gesture or overt behaviour showing the team acting upon a decision). Despite the fact that teams were 
provided with feedback based on specific criteria, improvement strategies defined by teams could still have 
been too general or abstract to be directly put into action, or coordination problems could have render the 
well-defined plans unused. A further investigation of implementation strategies and their quality seems 
warranted. 
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Overall, this empirical study was designed to meet methodological requirements of a rigorous design 
with high internal validity, based on its temporal sequencing and the collection of objective performance 
data. Capturing fundamental processes in a controlled environment is a first step to better understand 
complex phenomena. To that purpose, we used a flight simulation as a research platform to simulate and 
control task and team features. Simulations constitute an interesting environment that offers standardised 
performance measures with possibilities of controlling complexity, information overload, and cues available 
from the environment (Mathieu, 2000). In a learning perspective, they also allow learners to apply their 
knowledge and understanding to a task and observe the effects of their decisions in a reactive environment 
that offers real-time feedback (Bronack, Riedl, & Tashner, 2006; Gredler, 2004). Prior work on simulations 
has shown that they stimulate numerous cognitive processes, such as higher-level reasoning or creative 
thinking (Moreno & Mayer, 2005; Moreno, Mayer, Spires, & Lester, 2001). As a trade-off, laboratory 
environments overlook natural factors in real-world contexts that may mediate learning. As such, the extent 
to which the results from our controlled design can be generalised to real had-hoc teams has to be considered 
with cautious. The artificial and temporal nature of the team and the limited number of team members must 
be acknowledged. Teams of more than two team members and/or knowing each other before completing a 
team task might exhibit more complex interaction patterns. In this regard, we acknowledge that there has 
been a recent debate about whether findings from research on dyads can be simply generalised to larger 
teams (Moreland, 2010; Williams, 2010). We are aware that certain aspects of team processes and dynamics 
(e.g., group socialization) can hardly be grasped by the use of dyads and that the addition of team members 
increases complexity of team communication and coordination (Michinov & Michinov, 2009; Noroozi et al. 
2012). However, since research into the dynamic aspects of co-reflection in teams is a relatively new area, 
the present study looking into the timing of basic team behaviours in the smallest form of teams provides a 
very good start for further research. A replication study with triads and larger groups is needed to corroborate 
our results and explore the relationship between team size and successful team reflexivity. Moreover, 
although novelty of the task was controlled, we did not account for group-ability composition while research 
has demonstrated its influence on the accuracy and quality of explanations in teams (Webb et al., 1998). 
Finally, the use of students is sometimes considered as a possible limitation. Nevertheless, previous studies 
have established that little difference holds between the use of students and professional teams when using 
problem solving and decision-making scenarios (Balijepally et al., 2009; Yoo & Alavi, 2001). Still, further 
research exploring the effects of reflexivity with explicit feedback with more team members and in different 
settings will be needed to understand the complexity of how team members with different expertise, 
knowledge, and possibly high diversity deal with feedback that describes the aggregated group effort 
towards a shared goal. Furthermore, future field studies obviously need to consider critical contextual factors 
that influence and constraint team behaviours (Kozlowski & Ilgen, 2006). 

Another limitation of the study is the relatively lower frequency of some reflective behaviours (e.g., 
looking for alternatives). Similarly, sequences of reflective behaviours did not occur very often, limiting the 
possible analyses relating these to performance improvements. This necessitates caution about drawing 
premature conclusions and underlines the need for replication studies with larger samples. Additionally, this 
limitation could be overcome in future research by stimulating or training teams to become reflexive (e.g., 
King, 1991) and comparing occurrences of reflective behaviours and their relation with team performance 
with a no- training condition. Another challenging issue is motivational: we do not know why teams did not 
frequently reflect after feedback. Motivational factors behind a lack of reflection and receptivity to feedback 
should be investigated in further research. 

Finally, the relation between team performance (signified by accurate and specific feedback) and 
team reflexivity should be analysed in longitudinal designs. Additional measurement points of team 
reflexivity spaced over time would provide a more fine-tuned understanding of under which circumstances 
previous performance has more impact on learning. Feedback loops in which previous performance acts as 
an input for determining subsequent processes and performance have been recently forwarded as relevant 
models to understand team dynamics (Ilgen et al., 2005). 
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7. Practical implications 

The present study suggests that to be effective, feedback requires high levels of cognitive 
engagement from learning teams. However, discussion of and reflection on underlying reasons for success or 
failure, alternatives, and improvement strategies does not seem to happen spontaneously in teams, which in 
turn brings out the need to provide them with appropriate external support. This has potential implications 
for education. First, teachers should uncover whether a lack of engagement in thoughtful analysis of team 
experience is due to a lack of ability to perform shared reflection (i.e., availability deficiency) or a lack of 
execution of available skills (i.e., production deficiency). If students know how to reflect as a team, prompts 
(i.e., scenarios indicating how learners should interact) designed to induce inferences and deep-oriented 
processing of the content of the feedback appear an appropriate intervention to enhance teams’ motivation to 
engage in team reflexivity and to elicit learning strategies which teams would not naturally demonstrate 
(Veenman & Elshout, 1999). If students have not yet acquired the skills needed to perform shared reflection, 
teachers may first model and organise repeated practice with reflective cycles and provide more guidance 
and structure in their prompts (King, 2007). However, it may be that when students have attained a high 
level of reflective skills, prompts need to be withdrawn or faded-out to facilitate internalization (Dillenbourg 
& Tchounikine, 2007). Second, teachers should plan time and space for feedback, in which errors are 
considered as learning opportunities, and provide tools enabling students to actually perform reflective 
activities on feedback (Gan & Hattie, 2014). 


Keypoints 


•• There is a lack of evidence in team and collaborative learning research on the role of team 
reflexivity when team performance feedback is provided. This empirical (explorative) study 
provides a better understanding of how teams actively process feedback they receive and thus 
collaboratively evaluate information about past activities and derive better solutions for next 
action and better team performance 

•• Theoretically, team reflexivity is often seen as a process consisting of a series of steps, 
including evaluating performance and strategies, looking for alternatives, and making a clear 
decision about how to implement changes. However, no empirical work has confirmed this 
sequential view. Our results show that teams seldom engage in full reflective cycles following 
feedback reception 

When looking into reflective behaviours individually, our study shows that teams that analyse 
their performance and strategies and underlying reasons for success or failure during feedback 
(after action) are able to improve their subsequent team performance. By contrast, it seems that 
hasty decision-making (occurring at the beginning of team interaction) might be detrimental to 
future performance 

We question the fundamental idea of one-way causality between team reflexivity and team 
performance. While we find instances of better subsequent performance after mutual reflection 
upon feedback, we also find that initial mistakes promote team reflexivity 
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